Definition: Let (V,F) be a vector space. A norm on V is a function
∥⋅∥:V→R≥0
such that for all x,y∈V and c∈F the following axioms hold:
(P1) ∥x∥≥0 and ∥x∥=0 iff x=0v (positive definiteness)
(P2) ∥cx∥=∣c∣∥x∥∀c∈R (homogeneity)
(P3) ∥x+y∥≤∥x∥+∥y∥ (triangle inequality)
The triplet (V,F,∥⋅∥) is called a normed linear space.
Remark: Perhaps the most important property of a norm is that it induces a metric on the vector space. Hence we can quantify the distance between two vectors in a vector space. Namely, the distance between two vectors x and y∈V is the norm of the vector x−y or y−x:∥(y−x)∥. Since the norm is always positive, the distance is also positive. The distance between two vectors is zero iff the vectors are the same. x=x−0, the norm of x is the distance of x to the origin. With a proper distance definition (norm), one can begin studying the geometry of the space.
Remark: We can define a "sphere" in V using the norm concept. The sphere is the set of all vectors in V with a fixed norm. For example, in R2, the sphere with radius r is the set of all vectors with norm r.
S=v∈V∣∥v−v0∥≤r
Example: Let V=R2, F=R, and let x=[x1x2]
i. ∥x∥1:=∣α1∣+∣α2∣, is ∥x∥1 a norm?
(P1) Let x=[c1c2],
then ∥x∥1=∣c1∣+∣c2∣≥0 and ∥x∥1=0 iff c1=c2=0 (forwards direction)
(P2) Let c∈R, then ∥cx∥2=(cx1)2+(cx2)2=∣c∣x12+x22=∣c∣∥x∥2
(P3) Let y=[y1y2],
then ∥x+y∥2=(x1+y1)2+(x2+y2)2≤x12+y12+x22+y22=∥x∥2+∥y∥2
iii.
All these norms can be generalized into what is called the p-norm.
∥x∥p=(i=1∑n∣xi∣p)1/p
where p≥1 and x=⎣⎡x1x2⋮xn⎦⎤
Remark: Note that limp→∞∥x∥p=∥x∥∞=maxi∣xi∣
Geometric visualization of the p-norms in R2 is given below.
Example: Let ∥x∥21=(∣α1∣1/2+∣α2∣1/2)2 where, x=[α1α2] is ∥x∥21 a norm ?
Show a counter example for (P3),
Let x=[10] and y=[01]
then ∥x+y∥21=(∣1∣1/2+∣1∣1/2)2=4
but ∥x∥21+∥y∥21=(∣1∣1/2+∣0∣1/2)2+(∣0∣1/2+∣1∣1/2)2=2
hence ∥x+y∥21≰∥x∥21+∥y∥21
Example: Norms on function spaces. V:{f(.)∣f[0,1]→R s.t. ∫01f(t)pdt<∞,1≤p<∞}
∥f∥p=(∫01∣f(t)∣pdt)p1 (p-norm)
∥f∥∞=maxt∈[0,1]∣f(t)∣ (infinity-norm)
Matrix Norms
Definition: Let A∈Rm×n be a matrix. A matrix norm is a function that maps matrices to non-negative real numbers. A matrix norm must satisfy the norm axioms. A norm on matrices can be defined as,
∥A∥=ijmax∣aij∣
where aij is the element of A at the i th row and j th column.
Example: Let V=Rn×m and A=[aij]
∥A∥1=1≤j≤mmaxi=1∑n∣aij∣(absolute sum of rows)
Another norm definition is Frobenuis norm:
∥A∥F=(i=1∑nj=1∑m∣aij∣p)p1where 1≤p≤∞
Induced Norm
Definition: A:Rn→Rm be an m×n matrix. Let ∥⋅∥Rn be a norm on Rn and ∥⋅∥Rm be a norm on Rm. The norm of A induced by these norms is defined as,
∥A∥:=x∈Rnmax∥x∥Rn∥Ax∥Rm
Remark: The induced matrix norm is defined in terms of vector norms. An equivalent definition is given below:
∥A∥:=∥x∥Rn=1max∥Ax∥Rm
Remark: The induced norm of a matrix is the maximum amplification of the norm of a vector under the action of the matrix. In other words, the induced norm of a matrix is the maximum amount by which the matrix can stretch a vector.
A nice visualization of the induced norm is given below.